A Simple Way to Estimate the Cost of Downtime

It is time for the systems community of researchers and developers to broaden the agenda beyond performance. The 10000X increase in performance over the last 20 years means that other aspects of computing have risen in relative importance. We see three challenges for the future as [Patterson2002a]: 1. Synergy with humanity: We need to make the technology match human nature, both for users of personal gadgets and for those who operate the services accessed by those gadgets. Thus, we must care as much about the human cost of operating information technology as about the cost of purchasing it. 2. Dependability: We need to create a technology that the world can really depend upon, since its already doing it with a technology that does not deserve our trust. 3. Security/Privacy: We need to invent the technologies that helps make our societies secure without sacrificing privacy. Although these goals cover all computing devices, we are especially interested in services over a network, which covers both Internet sites and enterprise data centers. The systems we have created are fast and cheap, but undependable. To understand why they are undependable, we conducted two surveys on the causes of downtime, with unexpected results [Patterson 2002b]. In our first survey, we collected failure data on the U.S. Public Switched Telephone Network (PSTN). In our second, we collected failure data from three Internet sites. Based on that data, Figure 1 shows the percentage of failures due to operators, hardware failures, software failures, and overload. The surveys are notably consistent in their suggestion that operators are the leading cause of failure. The high fraction of operator error suggests that little attention has been paid to making systems easy to operate. This figure inspired the Recovery Oriented Computing (ROC) project [Patterson 2002b]. It takes the perspective that hardware faults, software bugs, and operator errors are facts to be coped with, not problems to be solved. By concentrating on Mean Time to Repair (MTTR) rather than Mean Time to Failure (MTTF), ROC reduces recovery time and thus offers higher availability. Since a portion of system administration is dealing with failures [Anderson 2000], ROC may also reduce total cost of ownership. The ROC project is investigating several techniques to reduce the consequences of operator error. Improving dependability and lowering cost of ownership is likely to require more resources. For example, an undo system for operator actions would likely need more disk …